Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made both neurobiologically more plausible and computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, for example, fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of recurrent neural networks may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics

Distortions of Subjective Time Perception Within and Across Senses

Background: The ability to estimate the passage of time is of fundamental importance for perceptual and cognitive processes. One experience of time is the perception of duration, which is not isomorphic to physical duration and can be distorted by a number of factors. Yet, the critical features generating these perceptual shifts in subjective duration are not understood. Methodology/Findings: We used prospective duration judgments within and across sensory modalities to examine the effect of stimulus predictability and feature change on the perception of duration. First, we found robust distortions of perceived duration in auditory, visual and auditory-visual presentations despite the predictability of the feature changes in the stimuli. For example, a looming disc embedded in a series of steady discs led to time dilation, whereas a steady disc embedded in a series of looming discs led to time compression. Second, we addressed whether visual (auditory) inputs could alter the perception of duration of auditory (visual) inputs. When participants were presented with incongruent audio-visual stimuli, the perceived duration of auditory events could be shortened or lengthened by the presence of conflicting visual information; however, the perceived duration of visual events was seldom distorted by the presence of auditory information and was never perceived shorter than their actual durations. Conclusions/Significance: These results support the existence of multisensory interactions in the perception of duration and, importantly, suggest that vision can modify auditory temporal perception in a pure timing task. Insofar as distortions in subjective duration can neither be accounted for by the unpredictability of an auditory, visual or auditory-visual event, we propose that it is the intrinsic features of the stimulus that critically affect subjective time distortions

Inhibition decorrelates visual feature representations in the inner retina

The retina extracts visual features for transmission to the brain. Different types of bipolar cell split the photoreceptor input into parallel channels and provide the excitatory drive for downstream visual circuits. Mouse bipolar cell types have been described at great anatomical and genetic detail, but a similarly deep understanding of their functional diversity is lacking. Here, by imaging light-driven glutamate release from more than 13,000 bipolar cell axon terminals in the intact retina, we show that bipolar cell functional diversity is generated by the interplay of dendritic excitatory inputs and axonal inhibitory inputs. The resulting centre and surround components of bipolar cell receptive fields interact to decorrelate bipolar cell output in the spatial and temporal domains. Our findings highlight the importance of inhibitory circuits in generating functionally diverse excitatory pathways and suggest that decorrelation of parallel visual pathways begins as early as the second synapse of the mouse visual system

Getting warmer: predictive processing and the nature of emotion

No description supplie

Increased sporulation underpins adaptation of Clostridium difficile strain 630 to a biologically–relevant faecal environment, with implications for pathogenicity

Abstract Clostridium difficile virulence is driven primarily by the processes of toxinogenesis and sporulation, however many in vitro experimental systems for studying C. difficile physiology have arguably limited relevance to the human colonic environment. We therefore created a more physiologically–relevant model of the colonic milieu to study gut pathogen biology, incorporating human faecal water (FW) into growth media and assessing the physiological effects of this on C. difficile strain 630. We identified a novel set of C. difficile–derived metabolites in culture supernatants, including hexanoyl– and pentanoyl–amino acid derivatives by LC-MSn. Growth of C. difficile strain 630 in FW media resulted in increased cell length without altering growth rate and RNA sequencing identified 889 transcripts as differentially expressed (p < 0.001). Significantly, up to 300–fold increases in the expression of sporulation–associated genes were observed in FW media–grown cells, along with reductions in motility and toxin genes’ expression. Moreover, the expression of classical stress–response genes did not change, showing that C. difficile is well–adapted to this faecal milieu. Using our novel approach we have shown that interaction with FW causes fundamental changes in C. difficile biology that will lead to increased disease transmissibility

Predictive coding and thought

Predictive processing has recently been advanced as a global cognitive architecture for the brain. I argue that its commitments concerning the nature and format of cognitive representation are inadequate to account for two basic characteristics of conceptual thought: first, its generality--the fact that we can think and flexibly reason about phenomena at any level of spatial and temporal scale and abstraction; second, its rich compositionality--the specific way in which concepts productively combine to yield our thoughts. I consider two strategies for avoiding these objections and I argue that both confront formidable challenges

Spike-Based Bayesian-Hebbian Learning of Temporal Sequences

Many cognitive and motor functions are enabled by the temporal representation and processing of stimuli, but it remains an open issue how neocortical microcircuits can reliably encode and replay such sequences of information. To better understand this, a modular attractor memory network is proposed in which meta-stable sequential attractor transitions are learned through changes to synaptic weights and intrinsic excitabilities via the spike-based Bayesian Confidence Propagation Neural Network (BCPNN) learning rule. We find that the formation of distributed memories, embodied by increased periods of firing in pools of excitatory neurons, together with asymmetrical associations between these distinct network states, can be acquired through plasticity. The model's feasibility is demonstrated using simulations of adaptive exponential integrate-and-fire model neurons (AdEx). We show that the learning and speed of sequence replay depends on a confluence of biophysically relevant parameters including stimulus duration, level of background noise, ratio of synaptic currents, and strengths of short-term depression and adaptation. Moreover, sequence elements are shown to flexibly participate multiple times in the sequence, suggesting that spiking attractor networks of this type can support an efficient combinatorial code. The model provides a principled approach towards understanding how multiple interacting plasticity mechanisms can coordinate hetero-associative learning in unison